On the Creative Use of Score Following and Its Impact on Research

(Abstract to follow

Modélisation anticipative des systèmes musicaux. Reconnaissance, génération, synchronisation et programmation synchrone temps réel en informatique musicale

National audienceThis paper studies musical anticipation as a design element for realtime computer music applications. Anticipatory modeling is presented as a design principle for modeling artificial systems. We propose anticipatory models and applications concerning three main preoccupations of expectation: What to expect? How to expect? and When to expect? Each question is addressed within different literatures such as information geometry, active and online learning, and stream synchronisation in realtime and for music signals. The anticipatory design concept is common in all propositions with the premise of reducing the structural and computational complexity for complex problems in computer music.Cet article étudie les apports de la modélisation de l'anticipation musicale à la conception d'application musicale temps réel. Nous proposons d'aborder le problème de l'anticipation sous l'angle de la notion d'attente et à partir de trois questions : quoi attendre ? Comment attendre ? Et quand attendre ? Ces questions sont formalisées à partir d'outils théoriques variés faisant appel respectivement à la géométrie de l'information, à l'apprentissage actif et automatique, et à la synchronisation de flux en temps réel. Notre objectif est d'intégrer ces modèles de l'attente comme des éléments de conception permettant de réduire les complexités structurelles et computationelles souvent exhibées par les applications musicales

L'ordinateur qui joue comme un musicien

National audienceCapable of live interactions with humans, Antescofo is a software capable of acting like a live musician on stage. It is capable of interpreting a music score and adapting itself to the interpretation of live musicians on stage.Doté de la capacité d'interaction propre à l'homme, le logiciel Antescofo est un musicien à part entière. Il est capable d'interpréter une partition tout en s'adaptant au jeu des instrumentistes

Synchronisme musical et musiques mixtes: du temps écrit au temps produit

International audienceMusical synchronism is essential to interpreting written music. Performance musicians are skilled in their capacity to achieve real time synchronization and coordination, with multiple musicians each playing their part to produce a coherent result. To successfully elicit a context of interaction between musicians and the machine raises interesting challenges for artificial intelligence and the practice of mixed music. This article explores musical synchronism in mixed music, looking at the practical, artistic and technological aspects, and presents recent and historical points of view drawn from subjects of musical research related to this question.Le synchronisme musical est essentiel à l'interprétation de la musique écrite. La capacité de synchronisation et de coordination en temps réel entre plusieurs musiciens sur scène, interprétant chacun sa propre partie et donnant un résultat d'ensemble cohérent, est un acquis commun des musiciens, qui pose des défis intéressants à l'intelligence artificielle ainsi que la pratique musicale de la musique mixte, afin de pouvoir la ramener dans un contexte d'interaction entre les musiciens et la machine. Cet article étudie la question du synchronisme musical dans la musique mixte, une problématique qui est présentée sous l'angle des aspects pratique, artistique et technologique, et expose des points de vue historiques et récents autour des sujets de recherches musicales connexes à cette problématique

A Coupled Duration-Focused Architecture for Real-Time Music-to-Score Alignment

International audienceThe capacity for realtime synchronization and coordination is a common ability among trained musicians performing a music score that presents an interesting challenge for machine intelligence. Compared to speech recognition, which has influenced many music information retrieval systems, music's temporal dynamics and complexity pose challenging problems to common approximations regarding time modeling of data streams. In this paper, we propose a design for a realtime music to score alignment system. Given a live recording of a musician playing a music score, the system is capable of following the musician in realtime within the score and decoding the tempo (or pace) of its performance. The proposed design features two coupled audio and tempo agents within a unique probabilistic inference framework that adaptively updates its parameters based on the realtime context. Online decoding is achieved through the collaboration of the coupled agents in a Hidden Hybrid Markov/semi-Markov framework where prediction feedback of one agent affects the behavior of the other. We perform evaluations for both realtime alignment and the proposed temporal model. An implementation of the presented system has been widely used in real concert situations worldwide and the readers are encouraged to access the actual system and experiment the results

Accelerating the Mixing Phase in Studio Recording Productions by Automatic Audio Alignemtnt

International audienceWe propose a system for accelerating the mixing phase in a recording production, by making use of audio alignment techniques to automatically align multiple takes of excerpts of a music piece against a performance of the whole work. We extend the approach of our previous work, based on sequential Montecarlo inference techniques, that was targeted at real-time alignment for score/audio following. The proposed approach is capable of producing partial alignments as well as identifying relevant regions in the partial results with regards to the reference, for better integration within a studio mix workflow. The approach is evaluated using data obtained from two recording sessions of classical music pieces, and we discuss its effectiveness for reducing manual work in a production chain

Coherent Time Modeling of semi-Markov Models with Application to Real-Time Audio-to-Score Alignment

International audienceThis paper proposes a novel insight to the problem of duration modeling for recognition setups where events are inferred from time-signals using a probabilistic framework. When a prior knowledge about the duration of events is available, Hidden Markov or Semi-Markov models allow the setting of individual duration distributions but give no clue about their choice. We propose two criteria of temporal coherency for such applications and prove they are fulfilled by statistical properties like infinite divisibility and log-concavity. We conclude by showing practical consequences of these properties in a real-time audio-to-score alignment experiment.Ce papier propose une nouvel éclairage sur la question de la modélisation des durées dans les algorithmes de reconnaissance, lorsque les événements reconnus sont inférés à partir de signaux temporels au moyen d'un modèle probabiliste. Si une connaissance a priori sur la durée nominale des événements est disponible, les modèles de Markov et de semi-Markov cachés permettent de choisir en fonction les distributions de durées de chaque événement, mais laissent ce choix complètement ouvert. Nous proposons deux critères de cohérence temporelle de tels algorithmes, et prouvons que ceux-ci si impliqués par des propriétés particulières étudiées en statistiques, telles que l'infinie divisibilité et la log-concavité. En conclusion, nous rapportons une expérience d'alignement audio-sur-partition en temps réel, qui montre l'intérêt pratique de ces propriétés théoriques

Non-overlapping, Time-coherent Visualisation of Action Commands in the AscoGraph Interactive Music User Interface

International audienceIntegrated authoring and performing of mixed music scores, where musicians interact dynamically with computer controlled electronics, is enabled by the Antescofo state-of-the-art software package. Composers are able to plan computerised actions through a dedicated programming language, and performances are then synchronised in real time. AscoGraph is the dedicated graphical interface that allows users to configure Antescofo behaviours and vi-sualise their layout over a mixed music score. This paper presents developments in the direction of increased clarity and coherence of AscoGraph's visualisation of com-puterised action scores. Algorithms for efficient automatic stacking of time-overlapping action blocks are presented, as well as a simplified model for displaying atomic actions. The paper presents the improvements in score readability achieved, as well as the challenges faced towards a complete representation of dynamic mixed scores in the Asco-Graph visual environment

On the Information Geometry of Audio Streams with Applications to Similarity Computing

International audienceThis paper proposes methods for information processing of audio streams using methods of information geometry. We lay the theoretical groundwork for a framework allowing the treatment of signal information as information entities, suitable for similarity and symbolic computing on audio signals. The theoretical basis of this paper is based on the information geometry of statistical structures representing audio spectrum features, and specifically through the bijection between the generic families of Bregman divergences and that of exponential distributions. The proposed framework, called Music Information Geometry allows online segmentation of audio streams to metric balls where each ball represents a quasi-stationary continuous chunk of audio, and discusses methods to qualify and quantify information between entities for similarity computing. We define an information geometry that approximates a similarity metric space, redefine general notions in music information retrieval such as similarity between entities, and address methods for dealing with non-stationarity of audio signals. We demonstrate the framework on two sample applications for online audio structure discovery and audio matching

Real-time detection of overlapping sound events with non-negative matrix factorization

International audienceIn this paper, we investigate the problem of real-time detection of overlapping sound events by employing non-negative matrix factorization techniques. We consider a setup where audio streams arrive in real-time to the system and are decomposed onto a dictionary of event templates learned off-line prior to the decomposition. An important drawback of existing approaches in this context is the lack of controls on the decomposition. We propose and compare two provably convergent algorithms that address this issue, by controlling respectively the sparsity of the decomposition and the trade-off of the decomposition between the different frequency components. Sparsity regularization is considered in the framework of convex quadratic programming, while frequency compromise is introduced by employing the beta-divergence as a cost function. The two algorithms are evaluated on the multi-source detection tasks of polyphonic music transcription, drum transcription and environmental sound recognition. The obtained results show how the proposed approaches can improve detection in such applications, while maintaining low computational costs that are suitable for real-time